Papers with disentangled representations
Knowledge Router: Learning Disentangled Representations for Knowledge Graphs (2021.naacl-main)
Copied to clipboard
| Challenge: | Existing approaches to learning from relational patterns and structural information ignore the intrinsic complexity of KGs. |
| Approach: | They propose to learn latent properties of KG entities by using a neighborhood mechanism to disentangle the inner properties of each entity. |
| Outcome: | The proposed method significantly improves performance on key metrics on several benchmark datasets. |
Counterfactuals to Control Latent Disentangled Text Representations for Style Transfer (2021.acl-short)
Copied to clipboard
| Challenge: | Existing methods for unsupervised text style transfer focus on transferring a specific attribute, but this technique has never been explored in natural language generation tasks. |
| Approach: | They propose a counterfactual-based method to modify latent representations by posing a ‘what-if’ scenario. |
| Outcome: | The proposed method is tested on multiple attribute transfer tasks like Sentiment, Formality and Excitement to support the hypothesis. |
Disentangling Representations of Text by Masking Transformers (2021.emnlp-main)
Copied to clipboard
| Challenge: | Large pretrained models such as BERT encode a range of features into monolithic vectors, providing strong predictive accuracy across downstream tasks. |
| Approach: | They explore whether it is possible to learn disentangled representations by identifying existing subnetworks within pretrained models that encode distinct, complementary aspects. |
| Outcome: | The proposed method disentangles sentiment from genre in movie reviews, toxicity from dialect in Tweets, and syntax from semantics. |
Learning Disentangled Representations for Natural Language Definitions (2023.findings-eacl)
Copied to clipboard
| Challenge: | Disentangling the encodings of neural models is a fundamental aspect for improving interpretability, semantic control and downstream task performance in Natural Language Processing. |
| Approach: | They propose to use syntactic and semantic regularities in textual data to provide models with both structural biases and generative factors. |
| Outcome: | The proposed model outperforms baselines on several qualitative and quantitative benchmarks and improves the results in the downstream task of definition modeling. |
Real-World Compositional Generalization with Disentangled Sequence-to-Sequence Learning (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing approaches to compositional generalization have been designed with semantic parsing in mind. |
| Approach: | They propose a disentangled sequence-to-sequence model which encourages more disentanglement and improves its compute and memory efficiency. |
| Outcome: | The proposed model improves generalization performance across existing tasks and datasets and a new machine translation benchmark. |
StylePTB: A Compositional Benchmark for Fine-grained Controllable Text Style Transfer (2021.naacl-main)
Copied to clipboard
Yiwei Lyu, Paul Pu Liang, Hai Pham, Eduard Hovy, Barnabás Póczos, Ruslan Salakhutdinov, Louis-Philippe Morency
| Challenge: | Existing methods for text style transfer focus on individual high-level semantic changes but do not offer fine-grained control of sentence structure, emphasis, and content. |
| Approach: | They propose a large-scale text style transfer benchmark with 21 fine-grained stylistic changes across atomic lexical, syntactic, semantic, and thematic transfers. |
| Outcome: | The proposed method allows modeling fine-grained changes as building blocks for more complex, high-level transfers. |
FC-TTS: Style and Timbre Control in Zero-Shot Text-to-Speech with Disentangled Speech Representations (2026.acl-long)
Copied to clipboard
| Challenge: | Recent advances in text-to-speech (TTS) have enabled accurate imitation of reference speech in terms of both speaking style and speaker timbre. |
| Approach: | They propose a zero-shot text-to-speech framework that enables disentangled control of style and timbre by conditioning on two distinct reference utterances. |
| Outcome: | The proposed framework achieves high-fidelity synthesis and competitive zero-shot naturalness while supporting consistent and independent manipulation of style and timbre. |
Learning Disentangled Textual Representations via Statistical Measures of Similarity (2022.acl-long)
Copied to clipboard
| Challenge: | Existing approaches to disentangle a sensitive attribute from textual representations require training and multiple parameter updates. |
| Approach: | They propose a family of regularizers for learning disentangled representations that do not require training. |
| Outcome: | The proposed regularizers are faster and faster and achieve better results when combined with pretrained and randomly initialized text encoders. |
A Multi-Task Approach for Disentangling Syntax and Semantics in Sentence Representations (N19-1)
Copied to clipboard
| Challenge: | Empirically, the model with the best performing syntactic and semantic representations gives rise to the most disentangled representations. |
| Approach: | They propose a generative model that uses latent variables to learn a sentence that uses both latent and latent representations. |
| Outcome: | The proposed model achieves better disentanglement between semantic and syntactic representations by training with multiple losses, including losses that exploit aligned paraphrastic sentences and word-order information. |
Disentangled Sequence to Sequence Learning for Compositional Generalization (2022.acl-long)
Copied to clipboard
| Challenge: | Existing models struggle to generalize to unseen compositions of seen components . a new approach allows for disentangled representations and better generalization . |
| Approach: | They propose an extension to sequence-to-sequence models which encourage disentanglement by re-encoding source input. |
| Outcome: | The proposed extension delivers better generalization and more disentangled representations . human expressions can be understood by combining known atomic components . |
Disentangling Generative Factors in Natural Language with Discrete Variational Autoencoders (2021.findings-emnlp)
Copied to clipboard
| Challenge: | Disentangled representation learning aims to provide an interpretable representation of latent features and a framework for controlling the change of specific features. |
| Approach: | They propose a Variational Autoencoder based method which models language features as discrete variables and encourages independence between variables for learning disentangled representations. |
| Outcome: | The proposed model outperforms baselines on several qualitative and quantitative benchmarks and on a text style transfer downstream application. |
Disentangling Categorization in Multi-agent Emergent Communication (2022.naacl-main)
Copied to clipboard
| Challenge: | Recent work on the emergence of language between artificial agents has not isolated the effect of categorization power on inter-communication ability. |
| Approach: | They propose to use disentangled representations to quantify categorization power of agents to enable differential analysis between combinations of heterogeneous systems. |
| Outcome: | The proposed method reduces signaling accuracy by 40% despite encouraging compositionality in the artificial language. |
Are “Undocumented Workers” the Same as “Illegal Aliens”? Disentangling Denotation and Connotation in Vector Spaces (2020.emnlp-main)
Copied to clipboard
| Challenge: | popular pretrained models encode both denotation and connotation as one entangled representation . a researcher using a pretrained representation can confuse words with connotations . |
| Approach: | They propose a nerual netowrk that decomposes a pretrained representation as independent denotation and connotation representations. |
| Outcome: | The proposed model improves document rankings by comparing denotation and connotation representations with extrinsic representations. |
Graph Neural News Recommendation with Unsupervised Preference Disentanglement (2020.acl-main)
Copied to clipboard
| Challenge: | Existing methods to learn informative user and news representations fail to consider high-order connectivity underlying the user-news interactions. |
| Approach: | They propose a novel Graph Neural News Recommendation model with Unsupervised Preference Disentanglement which can encode high-order relationships into user and news representations by information propagation along the graph. |
| Outcome: | The proposed model can encode high-order relationships into user and news representations by information propagation along the graph and disentangle latent preference factors by a neighborhood routing algorithm. |
CCSRD: Content-Centric Speech Representation Disentanglement Learning for End-to-End Speech Translation (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Existing speech-to-text translation models can extract features from speech inputs, but they may include non-linguistic speech factors such as pitch, timbre and speaker identity. |
| Approach: | They propose a content-centric speech representation disentanglement learning framework for speech translation that decomposes speech representations into content representations and non-linguistic representations via representation disentanglement learning. |
| Outcome: | The proposed framework outperforms state-of-the-art speech translation models and cascaded models on five translation directions. |
A Novel Estimator of Mutual Information for Learning to Disentangle Textual Representations (2021.acl-long)
Copied to clipboard
| Challenge: | Existing methods for learning disentangled representations of textual data are difficult to implement and suffer from the degeneracy of other losses in multi-class scenarios. |
| Approach: | They propose a variational upper bound to the mutual information between an attribute and the latent code of an encoder that controls the approximation error. |
| Outcome: | The proposed method is superior on fair classification and on textual style transfer tasks. |
PRISM: Probabilistic Reward Model with Inherent Structural Modeling (2026.acl-long)
Copied to clipboard
| Challenge: | Existing evaluators compress diverse human judgments into a single scalar, leading to brittle alignment and reward hacking. |
| Approach: | They propose a Gaussian-based reinterpretation of reward evaluation as a conditional distribution and a mixture of Gaussians to capture conflicting preference dimensions. |
| Outcome: | The proposed model outperforms scalar baselines in accuracy and generalization. |